1. Field of the Invention
The present invention relates to the field of emotive content in electronic communication. More particularly, the present invention relates to methods of embedding and encoding emotive content into files and data-streams for transmission, decoding and parsing of these files and data-streams containing emotive content in communications.
2. Background
Emotions and computers have, almost from the inception of these devices, been diametrically opposed concepts. Where computers calculate, humans reason. Reasoning has the emotive component that has heretofore evaded computer models and theories. Feelings are separate and different from thinking and vice versa. The intellect has little room for irrational thought. Emotions and are tolerated as a human shortfall in an intellectuals character. Even in areas such as law, the legal mind is taught to avoid inflaming the passions of the jury. The teaching is to use cold logic in deducing, deriving and proving legal conclusions. In science, there is little room for emotion, as they tend to be unique in situation and circumstance. Emotions are not repeatable as scientific testing requires and elusive in most practical respects. For these and other reasons, computer programs, communication and software applications have tended to ignore emotions, not the least of which is that they are fleeting and too difficult to capture with any fidelity in an electronic format. We are emotional social creatures and we are known to use emotive capabilities at times unconsciously or subtly, but in many surprisingly useful and in powerful ways.
Transmission of feelings has heretofore been left mostly to the arts. However, textual communication has begun to find small niches to allow transmission of feelings. Emocons and smiley face graphic sets have emerged but these are still very primitive in allowing any precision in emotive transmission and certainly nothing practical in information processing technology. What are needed are techniques and mechanisms to facilitate the transmission and processing of emotive content. Content which would facilitate a richer communication alphabet and grammar as well as electronic tools for processing the emotive and cognitive content in electronic communications.
Embedded Emotive Content
Emocon type face glyphs are emerging in some communications as an attempt to add something to the drab text that generally carries the communication to readers. Current face glyphs are a small step up from emocons, facial expressions using ASCII code characters. With graphical output options currently available on computers and computing devices, facial glyph expressions have evolved from ASCII display character :-) faces onto face glyphs smiley faces. This trend to display graphical output on computer device screens and video is growing. Picture messaging and content delivery services are now available to mobile data computing devices. Companies are expanding support for all popular message devices and a variety of graphics-capable handheld and messaging computer devices. Although the hardware capabilities have enabled graphical face glyphs, these capabilities have not been used to expand capabilities in basic communication. The new graphics-capable devices do not have comparable new and better ways of communication that can take advantage of the enhanced technology. There is a need to increase the quality of basic textual communication. Textual communication has been in existence for a long time despite the advances in transmission and display technologies, there has been little advancement in functionality of textual communication above what has been known. What is needed are ways of increasing the precision and enriching textual communication without adding the burden of larger working vocabularies upon users.
Textual communication by its nature can be ambiguous or obscure because of the multiple meanings in language. Emotive content from gestures, facial expressions, body language, and such have been used in conjunction with words to transmit meaning in language. Emotions are mostly absent with modern communication relying on text to carry the message. However, emotive content is used to focus the text and words onto the intended meaning out of a number of possible alternative inferences. Methods for embedding emotive content by a sender have been developed (see U.S. patent application '624). These methods can be used to add emotive content by both graphical and textual mean. When emotive content is purely textual, understanding is further complicated for reasons such as imprecision in emotive intensity or ambiguity for lack of a more complete description of the emotive state. If emotive content is presented graphically, complexity is increased because the receiver is subjected to overtly emotional component much harder to ignore and impossible to decypher. Here, the receiver must “search themselves for an answer” or understanding. As it is now, the receiver cannot take advantage of computers to generate complete messages. Messages which ordinary human behavior and psychological principles are used to unravel the intended meaning in responding to received verbal messages. Hence, an entire band of communication lies dormant and unused in textual communication, often times leaving little understood and counter acting communications.
Moreover, available techniques are mostly not applied in the textual communication arena. There is extraordinary need for automated methods to provide for a richer, more complete communication between individuals.
Facial Features and Emotive Content
Facial features are an excellent method of transmitting emotive content. In terms of recognition, faces are not simply body parts, like the eyes and nose. Instead, the face is composed of tectonic plate like pieces. Pentland's (Arbitron—MIT Media Lab) research at MIT led to a device for recognizing faces. He compiled a face database, a “facebase”. He discovered a mosaic of about 100 pieces, which he called “eigenfaces”, independent units. Using these categories, Pentland found he could describe anyone in his database, even though there were at least 10^200 possible human faces.
Pentland claimed that “face recognition” or more generally interpreting people's faces, is a critical part of making machines, rooms, and cars be human-centric rather than technology-centric. Faces all convey meaning, sometimes in concert with other facial features or human attributes.
The human face, aside from all the other things it does, provides an incredible ability to communicate. We can convey an enormous range of feelings and thoughts through our faces. The raising of an eyebrow, the myriad number of looks in our eyes or the pursing of the lips can express so plainly, what our emotions are at any particular time. In a universe of faces, we can recognize a known one instantly and the transmitted emotions as well. Moreover patent application '624 offers a defined alphabet or emotive lexicon set for the transmission of emotive content via face glyphs and other methods, which can be used for reliable and precise textual communication.
Methods have been developed in '624, which use vectorized face glyphs to carry emotive content with text in addition to the conventional methods of emotive word and expression usage. These methods have not been exploited through emotive parsers in communications, response communications, profiling, decision-making, research, marketing, entertainment and many other applications which can make use of the emotive layer in communication. What is needed is an emotive lexicon set or “emotive alphabet” of faces for use in text, audio and video applications, which can facilitate transmission of emotive content between the various applications.
Response to Emotive Content
Some individuals may not respond to the embedded textual or graphical emotive content for any number of reasons. Some receivers may not wish to address the emotive content because it makes them uncomfortable, they may not recognize that they are receiving emotive content or only partial perceive it. Receivers may not have the language skills to formulate their response. They may not respond with emotive content out of fear or confusion. Receivers also may know that they respond automatically and wrongly giving them added hesitation or reluctance to attempt to identify and address emotive content. At times, relationships can get stilted or strained when communicating without emotion. The response to very subtle messages can be vital to a personal or business relationship. A perfunctory response which ignores or does not address portions of the received message can not only strain the relationship but may even be fatal. The objective in a response should be to restore understanding no matter how subtle or terse the message. Ways are needed to provide understanding in a more comprehensive, positive, cooperative manner.
Intended Meaning or Substantive Truth
At times the most important part of the message is unsaid or unclear. The listener relies on emotive content to interpret the message. Relationships are richer if this communication layer is used. Upon the death of the last California Indian, Ishii, one of his closest friends, Mr. Pope, wrote:                And so stoic and unafraid, departed the last wild Indian of America. He closes a chapter in history. He looked upon us as sophisticated children, smart but not wise. We know many things, and much that is false. He knew nature, which is always true. His were the qualities of character that last forever. He was kind; he had courage and self-restraint, and though all had been taken from him, there was no bitterness in his heart. His soul was that of a child, his mind that of a philosopher.        
Ishii was befriended and studied by Professor Kroeber, a Berkeley Anthropologist. During this time they exchanged language, history and culture as they learned to communicate with each other. How were the traits of “stoic” “unafraid” “wise” “kind” “courageous” “no bifterness” and “soul” transmitted to the people that lived with and studied Ishii? The language between Ishii and his new friends had to be learned and developed in a short time and therefore was never very good but on some level Ishii's communications and meanings were well understood and his character admired by his late found friends. Mr. Pope was even able to describe Ishii's soul in the passage above. These are observations are of substantial weight despite poor express language exchanged between parties. It is doubtful that we can achieve these types of insights and understanding through purely textual or verbal expression. The literature is replete with evidence which indicates that the quality of communication supercedes the verbal spoken or actual text of the communication. This kind of communication is richer in meaning and is almost innate to humans. It allows people to decipher meaning from mere words and expressions. As the world is made smaller through the Internet, many Ishii-Kroeber relationships develop through electronic communication. What is needed are ways to transmit and methods of intentionally embedding precise emotive content. Also needed are ways of inferencing intended meaning and substantive truth from communication without the benefit of physical proximity or visual interaction.
Rogerian Response
People reason and computers compute. The difference in communication is immense. Computer response was studied by Joseph Weizenbaum, who coded ELIZA at MIT during the years 1964-1966. The ELIZA program consists of two stages. The first stage used a parser to extract information from what the user typed into the program input interface, while the second stage used a script to formulate a suitable reply. This reply based on text input by the user, gave the user the feeling that a human was actually involved. Weizenbaum developed a script for ELIZA, which simulated a Rogerian psychotherapist. Another implementation is commonly known as DOCTOR. This was intelligent, as a Rogerian psychotherapist attempts to draw the patient out by reflecting the patient's statements back to him, and to encourage the patient to continue in the conversation. Rogerian techniques have been applied to electronic text processing as well. These are accomplished through text parsing, string editing through subject manipulation with pre-stored string fragments and grammatical string fragments synthesized in the form of questions. These techniques are well known but have not been yet applied to emotive content in communications. What are needed are applications which enhance communication and which add “human” intelligence to electronic devices.
Extraction of Emotion
The need to extract or infer emotional state or personality of the user by some technological, electronic or mechanical means is seen as an important step in capturing emotive information from senders publishers or authors. However, this step is rendered unnecessary if senders publishers or authors themselves are able to introspect, recall and embed emotive content directly into textual communication. The functionality, which allows users themselves to embed emotive state and associated intensity along with their textual communication, is contained in application '624. An emotive content format and protocol needs to be established in order for receiver applications and parsers are able to tokenize and assemble these in some prescribed order otherwise known as a grammar. What is needed are methods and standards which facilitate the encoding, transmission, decoding, parsing and programming of emotive layer content.
Computer Understanding of Communication
Natural Language Parsers (NLP) abound and there are many types; morphosyntactic, autolexical, cognative, construction, functional, categorical, head-driven phase structure, integrational, link, neurocognitive, stratificational, transformational, tree adjoining, contrastive, dependency-based, learning DCG-type, functional, linguistic, minimal information, generative, probabilistic, role and reference, syntactic, unification, etc. Most NLP parsers can accomplish the basic rudiments of recognition of keywords, proper names or other parts-of-speech. Technical term candidates can be based on automatic content analysis that includes part-of-speech tagging, lexical and syntactic analysis, as well as the analysis of the position and distribution of words. Content analysis can produce such things as estimations of essential information in a document and the term-likelihood of the candidates. For example, a morphosyntactic analyzer can identify basic linguistic features, using a syntax with a full-scale dependency parser and can show how words and phrases are functionally related to each other.
There are many public domain and shareware NLPs. Conexor or Cogilex are commercial NLP vendors among others, which sell NLP software components.
Software designed to read data-streams and tokenize sentences using the various grammar models, simple syntactic structures and complex structures can be found in the literature, public domain, free ware, shareware and commercial business arena. Parsers using dependency approaches based on lexical representation and grammatical rules or the processing of discontinuous text, speech transcripts, incomplete sentences, technical texts are commercially available.
These software packages can be programmed to tag noun phrases (NP), most verb phrase combinations (VP), apposition of NPs, non-finite structures, passives, small clauses, expletives, binary branching, etc. Source code to these types of parsers can be obtained and adapted to a particular application without undue experimentation or development. Most NLPs can parse out and tokenize the noun phrases to identify the subject matter, tokenize verb phrases for actions, and aid in formulation of appropriate grammatical structures for tense and conjugation.
Most NLP deal in lexicons, syntax, grammar, parts of speech, word relationship hierarchy, etc. There is the notable absence of emotive context and emotive content in NLP. Commonsense Knowledge Base approach is the nearest that current NLP comes and is seen by most as a failure. This deficiency of NLP is very restrictive on communication and understanding of textual messages. Moreover, this limitation of textual communication can breed misunderstanding and distrust between people, as an important layer of communication is systematically filtered out and text messages without the emotive component are misinterpreted.
Emotive Markup Languages
Text To Speech (TTS) programs and speech markup languages such as SABLE, SSML, STML, SML and JSML adapted to carry emotive content are beginning to emerge in research environments. SML supports only a few emotions; anger, sadness, happiness and fear. The methods used are only at their infancy stages, cumbersome and in need of more comprehensive more encompassing models for encoding, transmitting representing and parsing emotions in electronic communications.
Emotive Markup Language attempts currently use, explicit emotional indicators, which are translated into emotive states. In those applications, emotive states are limited to less than 10 and there is no associated emotive intensity. There is no attempt to translate between emotive states, which are same or similar but use different names. Apart from the related references '624 and '758 there is no concept of emotive vectorization, emotive intensity associated with a directed emotive state, or emotive vectorization and associated text, nor the concept of emotive normalization to the author. What are needed are standards for implementing emotive vectorization, emotive normalization and for a more precision in emotive content in communication.
Intended Meaning
Language carries alternate meanings and words carry weight. Parsers currently do not discern the alternate meanings of language colored by emotive expression. Parsers do not currently weigh words but people do. What is needed are parsers which can parse a communication transmission with embedded emotive content along with the cognitive content, tag the emotive lexical tokens and retain the relationships to the associated text. What are needed are parsers that can promote alternate selectable grammars aligned with the various social and cultural protocols to identify and promote the otherwise filtered or obscured meaning in a communication. What is needed are parsers which can inference meaning from textual and emotive content communication for intended meaning to determine the author's intent despite linguistic ambiguity, ambiguity which tends not to exist where the communication is within physical proximity to enable the receiver to discern the associated emotive context within which the textual communication lives.
Computers currently are not programmed to reason. That is partially because they cannot weigh words or discern meaning. At times what is not said is more important than what is said or how it is said carries the meaning instead of what is said. These are components that are currently missing from electronic communication. Reasoning dictates that the reasoner incorporate thoughts, feelings and wisdom into understanding. Thoughts can be represented by textual symbols and feelings can also to a certain degree. Thoughts can also be represented by diagrams and numerical models, but feelings have not been so represented by any precise method for many reasons, the best reason is that it is too difficult to fathom much less accomplish. What are needed are methods to facilitate emotive representation, the “unsaid” or the “how said”, in conjunction with other communication streams so that richer and more comprehensive communication can transpire.
Emotive content carries an invisible dimension in intelligence, artificial or otherwise. What are needed are adequate methods and standards, which can accommodate textual, audio and video applications with connectable emotive as well as conventional content. What are needed are file and transmission formats, which can accommodate a standard emotive content layer in parallel with the current content such that the emotive meaning can be communicated when lacking physical proximity.